A company whose software and services are used to remotely administer and monitor large sections of the energy industry began warning customers last week that it is investigating a sophisticated hacker attack spanning its operations in the United States, Canada and Spain. Experts say digital fingerprints left behind by attackers point to a Chinese hacking group tied to repeated cyber-espionage campaigns against key Western interests.
The attack comes as U.S. policymakers remain gridlocked over legislation designed to beef up the cybersecurity posture of energy companies and other industries that maintain some of the world’s most vital information networks.
In letters sent to customers last week, Telvent Canada Ltd. said that on Sept. 10, 2012 it learned of a breach of its internal firewall and security systems. Telvent said the attacker(s) installed malicious software and stole project files related to one of its core offerings — OASyS SCADA — a product that helps energy firms mesh older IT assets with more advanced “smart grid” technologies.
The firm said it was still investigating the incident, but that as a precautionary measure, it had disconnected the usual data links between clients and affected portions of its internal networks.
“In order to be able to continue to provide remote support services to our customers in a secure manner, we have established new procedures to be followed until such time as we are sure that there are not further intrusions into the Telvent network and that all virus or malware files have been eliminated,” the company said in a letter mailed to customers this week, a copy of which was obtained by KrebsOnSecurity.com. “Although we do not have any reason to believe that the intruder(s) acquired any information that would enable them to gain access to a customer system or that any of the compromised computers have been connected to a customer system, as a further precautionary measure, we indefinitely terminated any customer system access by Telvent.”
The incident is the latest reminder of problems that can occur when corporate computer systems at critical networks are connected to sensitive control systems that were never designed with security in mind. Security experts have long worried about vulnerabilities being introduced into the systems that regulate the electrical grid as power companies transferred control of generation and distribution equipment from internal networks to so-called “supervisory control and data acquisition,” or SCADA, systems that can be accessed through the Internet or by phone lines. The move to SCADA systems boosts efficiency at utilities because it allows workers to operate equipment remotely, but experts say it also exposes these once-closed systems to cyber attacks.
Telvent did not respond to several requests for comment. But in a series of written communications to clients, the company detailed ongoing efforts to ascertain the scope and duration of the breach. In those communications, Telvent said it was working with law enforcement and a task force of representatives from its parent firm, Schneider Electric, a French energy conglomerate that employs 130,000 and has operations across the Americas, Western Europe and Asia. Telvent reportedly employs about 6,000 people in at least 19 countries around the world.
The disclosure comes just days after Telvent announced it was partnering with Foxborough, Mass. based Industrial Defender to expand its cybersecurity capabilities within Telvent’s key utility and critical infrastructure solutions. A spokesperson for Industrial Defender said the company does not comment about existing customers.
A page from an alert Telvent sent to customers about the malware left behind by the intruders.
In its most recent dispatch to customers impacted by the breach, dated Sept. 25, 2012, Telvent executives provided details about the malicious software used in the attack. Those malware and network components, listed in the photocopied Telvent communication shown here strongly suggest the involvement of Chinese hacker groups tied to other high-profile attacks against Fortune 500 companies over the past several years.
Joe Stewart, director of malware research at Dell SecureWorks and an expert on targeted attacks, said the Web site and malware names cited in the Telvent report map back to a Chinese hacking team known as the “Comment Group.”
In July, Bloomberg News published an in-depth look at the Comment Group and its many years of suspected involvement in deploying sophisticated attacks to harvest intellectual property and trade secrets from energy companies, patent law firms and investment banks.
That investigation looked at data gathered by a loose-knit group of 30 security researchers, who tracked the Comment Group’s activity over less than two months last year and uncovered evidence that it had infiltrated at least 20 organizations — “many of them organizations with secrets that could give China an edge as it strives to become the world’s largest economy. The targets included lawyers pursuing trade claims against the country’s exporters and an energy company preparing to drill in waters China claims as its own.”
Politicians in Congress and the Obama administration are becoming more vocal about accusing China and Russia of hacking U.S. computer networks for economic gain, espionage and other motives. But those accusations tend to ring hollow abroad, as Reuters recently observed: “U.S. standing to complain about other nations’ cyber attacks has been undermined, however, by disclosures that Washington, along with Israel, launched sophisticated offensive cyber operations of its own against Iran to try to slow that nation’s suspected quest for a nuclear weapon.” The malware alluded to in that Reuters piece — Stuxnet — was designed to attack specific vulnerabilities in SCADA systems known to be used in Iran’s uranium enrichment facilities.
Nevertheless, a mounting body of evidence suggests the involvement of one or two Chinese hacking groups in a host of high-profile corporate cyber break-ins over the past several years. Symantec Corp. reported earlier this month that a Chinese hacker group responsible for breaking into Google Inc in 2009 – an operation later dubbed Operation Aurora – had since launched hundreds of other cyber assaults, focusing on defense companies and human rights groups. Earlier this week, I detailed additional research on this front which showed espionage attackers often succeed in a roundabout way — by planting malware at “watering hole” sites deemed most likely to be visited by the targets of interest.
Domains connected to different cyber-espionage botnets typically trace back to one of two destinations in China, according to Dell SecureWorks.
A possible way forward for enterprises?
The Australian Defence Signals Directorate Cyber Security Operations Centre published, in 2011, a document entitled “Strategies to Mitigate Targeted Cyber Intrusions” available here:
http://www.dsd.gov.au/infosec/top35mitigationstrategies.htm
From the introduction:
“At least 85% of the targeted cyber intrusions that the Defence Signals Directorate (DSD) responded to in 2010 could have been prevented by following the first four mitigation strategies listed in our Top 35 Mitigation Strategies:
1. patch applications such as PDF readers, Microsoft Office, Java, Flash Player and web browsers
2. patch operating system vulnerabilities
3.minimise the number of users with administrative privileges
4. use application whitelisting to help prevent malicious software and other unapproved programs from running.
“The Top 35 Mitigation Strategies are ranked in order of overall effectiveness. Rankings are based on DSD’s analysis of reported security incidents and vulnerabilities detected by DSD in testing the security of Australian Government networks.
P.S. I posted the link to this document as it provides the best actionable recommendations I have seen to date.
two year old data with a 15% fail rate isn’t exactly encouraging.
of course I am being conditioned by having three of the four in place, with reason to believe the fourth would not be effective, and still getting hit.
experience also makes me wonder how tightly they define “targeted”, and how broad their sample population might be. There would be a lot of difference between the internal Ministry of Defense systems and their broad defense industrial base (at least, I would so hope!).
okay, gave the Australian doc a quick read. Apparently the population was all Aus gov systems the Signals Directorate reviewed. I’d expect that group to overall fall somewhat below the DIB in cyber sec quality.
Some good ideas in the list, but there seemed to be little concern about practicality of implementing the recommendations (not infrequent in outside consultants who have no idea what it means to walk in their clients’ moccasins).
I’ll have to drill into some of the better ones to see if they are in fact useful. If possible I’ll post results, but may not be able to do so.
Thanks for the pointer!
Don’t forget that these attackers only use as much skill as needed. They are going to expend a lot of energy if something as simple as a 6-month old Adobe vuln. will get them the same access.
I’ve seen that Australian report cited before.
It’s mostly a rehash of common sense mitigation – which, however, is likely to be useless against competent hackers. As Grant notes, these methods will just raise the bar from “script kiddie” to “half-way decent hacker.”
We’re back to my meme: “You can haz better security, you can haz worse security. But you can’t haz ‘security’. There is no security, Deal.”
Also note that in this case it has nothing to do with protecting the SCADA systems themselves. What was stolen here is the project file and data on the software offerings to be used in SCADA systems, i.e., intellectual property.
Personally, I view this concerted efforts by China to hoover up intellectual property in the West to be essentially just “leveling the playing field” – or perhaps payback for the British introduction of opium back in the day.
Also, in my view the notion that one can’t compete if your IP is stolen is an over-reaction. Companies should be concerned about execution rather than secrets. The fact that the Third World has cheaper labor really doesn’t mean much in the technology field.
In any event, the reality is that the situation isn’t going to change no matter how many “government policies” are written into law. The Third World IS going to catch up to the First World eventually, especially if the West continues to pitch trillions down the war chute in an attempt to stave that off.
Example: Iran has one of the fastest growing economies in the world, and one of fastest growing scientific communities in the world. The West’s answer: bomb it over some bogus “nuclear weapons program”.
Industrial espionage predates the Internet age by decades. I remember reading a book about it several decades ago. In one case, an industrial spy simply walked into a CEO’s office, on whose desk was a valuable document the spy knew was there. He simply picked up the document, whipped out a camera, took pictures of the document, then walked over to the window and tossed the camera to an accomplice waiting in the street. The CEO raised hell, but there was no proof of espionage once the camera disappeared…
The best solution to stolen IP is…produce more IP faster…and execute faster than your competitors…
“Also note that in this case it has nothing to do with protecting the SCADA systems themselves.
Don’t forget Stuxnet. There was an air gap protecting the SCADA system at the Natanz facility. Stuxnet was carried through the air gap on a USB stick, infected a Windows workstation and gained access to and control of the SCADA system resulting in significant damage to centrifuges.
Is there a lesson here? Even if one does not connect their SCADA system to the internet, there is still a risk.
Good old “sneaker net” strikes again! 🙂
Richard Steven Hack wrote:
“It’s mostly a rehash of common sense mitigation – which, however, is likely to be useless against competent hackers. As Grant notes, these methods will just raise the bar from “script kiddie” to “half-way decent hacker.
If the top 4, along with the remaining 31, mitigations are “common sense”, then why have so many organizations failed to implement just the top 4 alone, which includes application whitelisting? And why have so many organizations failed to utilize Windows group policy and use Internet Explorer’s Security Zones to whitelist essential websites as Trusted Sites (please see mitigation strategies 10/11 and reread Brian’s recent article on watering holes)? “Common sense” is not as common as you believe it to be.
Also, Grant stated that “these attackers only use as much skill as needed”. This is quite different from what you paraphrased. Here’s another link from the above-referenced site (see “Creating a defence-in-depth system”):
http://www.dsd.gov.au/publications/csocprotect/top_4_mitigations.htm
Specifically, the top four strategies “will help protect an organisation from low to moderately sophisticated intrusion attempts”. Thus, it will winnow out, as you have described, both the “script kiddies” and “half-way decent hackers”. Adding additional mitigations from the list, which btw include both prevention and detection capabilities, as appropriate for one’s organization, will make the work of advanced hackers more difficult. Why on God’s green earth make it easy for the miscreants?! Make them burn the midnight oil for months, or even years.
Finally, even if your “meme” is correct and one cannot prevent intrusions, this would imply that organizations should also implement strong detection capabilities to enable response to attacks at the earliest possible moment. As an example, shortly after RSA was hacked in March, 2011, an attack was launched against one of their customers, Lockheed Martin. Lockheed Martin is believed to have detected network intrusion attempts by the miscreants and, in response, shut down remote access and re-issued tokens and passwords. More here:
“Data Breach at Security Firm Linked to Attack on Lockheed
http://www.nytimes.com/2011/05/28/business/28hack.html?_r=0
Raising the white flag and innovating more quickly is not the answer.
Rabid Howler Monkey: “‘Common sense’ is not as common as you believe it to be.”
That was my point. Common sense mitigations are not being done and when they are done they will be bypassed anywhere.
“Also, Grant stated that “these attackers only use as much skill as needed”. This is quite different from what you paraphrased”
Not really. The point is that if the attacker is competent, common sense mitigations will not keep them out. You have to go BEYOND the industry “best practices” because such practices will eventually be bypassed.
The overall point is directly related to my meme: You are in a CONFLICT with an actual human being, not some robot. In fact, you are in conflict with a HORDE of human beings, some smarter than others. Unless you deal with the overall concept of security in this context, you are not going to win.
The majority of the infosec industry still thinks that computer security is some kind of “technology issue” to be solved by applying various bandaid “best practices.” It’s not. It’s a human conflict and the principles of conflict apply. In other words, you have to deal with the attack that happens and not the one you prepared for. This is a commonly known principle in martial arts and in the military (although in the military, it, too, is not usually performed correctly due to organizational inertia. Case in point: the utter failure of Afghanistan.)
“Specifically, the top four strategies “will help protect an organisation from low to moderately sophisticated intrusion attempts”. Thus, it will winnow out, as you have described, both the “script kiddies” and “half-way decent hackers”. Adding additional mitigations from the list, which btw include both prevention and detection capabilities, as appropriate for one’s organization, will make the work of advanced hackers more difficult. Why on God’s green earth make it easy for the miscreants?! Make them burn the midnight oil for months, or even years.”
They will never take years to penetrate a system, and most likely not even months.
Richard Marcinko’s Red Cell SEAL Team penetrated just about every US military security capability there was during his tenure as its commander. His team put IEDs next to the nuclear reactors on nuclear subs at Groton, put IEDs on Air Force One, and got several SEALS with several pounds of C-4 within 20 yards of the President’s cottage at Camp David.
Penetrating a computer security system is way easier than any of those operations.
It is literally – and I mean LITERALLY – IMPOSSIBLE to secure a company with over, say, 10-100 employees. Corporation with thousands of employees are Swiss cheese no matter what they do.
Besides, I never said one should NOT do the mitigations – I’m saying you cannot RELY on them to be effective.
“Finally, even if your “meme” is correct and one cannot prevent intrusions, this would imply that organizations should also implement strong detection capabilities to enable response to attacks at the earliest possible moment.”
With this I agree completely. Since you’re going to be attacked, and probably breached, you need to be able to detect when and where and head off the attackers before they can either do damage or acquire what they’re after.
“Raising the white flag and innovating more quickly is not the answer.”
Controlling your own game is much easier than controlling the enemy’s game. Every coach knows this.
Thanks for the link, that’s useful info. I work at a small business and went down the full Top 35 list. We implement 12 in full, including the top 4 with great rigorousness, and two more to a partial extent. I marked three more from their list for near-term implementation. If they were building that list today, I expect Microsoft EMET would be among the suggestions.
#4 on their list is application whitelisting (SRP or AppLocker). AppLocker has potential, but is only found in the Ultimate and Enterprise versions of Windows, so I’ve stuck with SRP.
I have a page on Software Restriction Policy at http://www.mechbgon.com/srp that I recently revamped after reviewing the United States NSA’s PDF on the subject, and Didier Stevens’ blog. Those of you who do use SRP (or plan to try it), should look at those resources, so here they are:
NSA: http://www.nsa.gov/ia/_files/os/win2k/Application_Whitelisting_Using_SRP.pdf
Didier Stevens:
http://blog.didierstevens.com/2011/11/17/hotfix-for-srpapplocker-bypass/
For those of you using Google Chrome, you’ll want to switch to the .MSI version of it so it gets installed into the Program Files directory, rather than into your user profile, which will become a no-execute zone under SRP’s protection. That version of the Chrome installer can be obtained at https://www.google.com/intl/en/chrome/browser/eula.html?msi=true
Good post! The last time I installed Chrome, it was already changing to the program folder. I remember having some minor hiccups at that time. I’m using the Comodo Dragon version now, and it is set the same way. No more need to do separate “installations”.
this coming just days after they announced their partnership with Industrial Defender makes me wonder if the new partner was the first to look at their logs with any kind of knowledgeable eyes.
most worrisome is the quote “…we indefinitely terminated any customer system access by Telvent.” what is the likelihood that the credentials used for that access have already been exploited to install persistent malware with permanent backdoors on all those customer systems?
all the patching and other mitigations recommended by the Rabid Howler won’t protect in that case, will they?
big ouch!
all the patching and other mitigations recommended by the Rabid Howler won’t protect in that case, will they?
Just imagine if Telvent had placed more emphasis on mitigation strategies for targeted attacks. Perhaps they and their clients would not be in the pickle that they are in today.
the hits just keep on coming. found this tidbit on the Telvent corporate site:
– In Europe and South America, we manage IT infrastructure for more than 350 companies through our state-of-the art data centers network.
I’m a bit concerned by the language in this paragraph:
“Security experts have long worried about vulnerabilities being introduced into the systems that regulate the electrical grid as power companies transferred control of generation and distribution equipment from internal networks to so-called “supervisory control and data acquisition,” or SCADA, systems that can be accessed through the Internet or by phone lines. ”
Its not the SCADA systems themselves that are the problem – they have been around most industrial plants since the 60’s and are the “internal networks” … but they assume they are the only network, and that there is no external access. It’s when someone decides to bolt on remote access to them without proper controls that the problem happens. IMHO.
Great reporting.
FYI, Telvent’s OASyS DNA has the majority market share for oil/gas pipeline SCADA in N. America and sizeable worldwide presence. This is there traditional market, but they have been expanding in recent years to electric sector and smart grid.
Dale Peterson
Digital Bond, Inc.
One of the immediate observations that cropped up on the SCADASEC e-mail reflector is that organizations such as Telvent have agreements to keep customer systems properly patched and up to date.
Therefore, if you wanted to plant your software right in to the middle of a customer’s system, all you have to do is to hack the company that integrated and maintained your SCADA system. From there you’d have access to all of their customers.
There is more to this story than meets the eye.
Jake, how much would having the intellectual property that was stolen help in the process?
I would imagine that the project files involved could be like a road map into each of the Telvent customers they describe, and any OASYS software source code would be like giving the key to drive a fully customized vehicle down that road.
We are so hosed…
There isn’t that much in the way of intellectual property to be concerned about. The stuff that hackers would want to attack (electric utilities, water utilities, transportation systems, and the like) really do operate out in the open. They are often governed by Public Utilities Commissions and as such their operation is a matter of public record.
What should scare everyone is the opportunity to issue nefarious controls at just the right time. Now to do that, you’d need intimate knowledge of not only how the SCADA system works, but how the electric distribution system works, how the water system works, how the transportation systems work –in other words, it’s not enough to be an IT geek. You have to have the experience of an engineer with significant detailed knowledge of the infrastructure as well.
Yes, this is security through obscurity. However, do note that the number of people on this planet who could pull off a successful attack like this are probably measured in the low hundreds. This combination of skill and experience is very unusual.
As for those of you who shake your heads at the choices made in SCADA systems, remember this: the goals (in order) are Safety, Availability, Integrity; and perhaps confidentiality, though aside of keeping keys and access systems secure, it doesn’t figure in to this. Many of the security assumptions you have learned from office applications are inappropriate in this context.
Also, sooner or later we will all slip up and someone will manage to hack us. All this doom about Telvent ignores the fact that they’ve been doing a pretty decent job of handling this situation. I am not a Telvent customer or a Telvent manager. I’m merely a SCADA user who is trying to deal with the reality of keeping a distribution system going.
Though I’m not happy about the fact that they did get hacked, I’m impressed that they 1) caught it, 2) did their best to remediate it, and 3) put their customers first. Contrast that with how Siemens handled Stuxnet.
HA! HA! I imagine this site is one of those “watering holes” mentioned in the article! :O
I’ve been a user of OASyS DNA for years. If the Chinese were trying to steal advanced intellectual property from Telvent, the joke is on them…
How many of their competitors have been pwned and don’t know it yet, or won’t admit it if they know it?
Reverse engineer them all, take the best and leave the rest, you end up with some decent features. Then, as Richard Steven Hack observed, it comes down to execution.
Whose execution is better, the successful intruders or the victims that can’t keep them out? Think of it as Darwinian selection in the free market economy…
Sounds like denying ISO/IEC 27000 family of standards was published in October 2005 by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC)? 🙁
And when the only ones left are the ones who stole technology from others and were therefore able to spend more of their time making the corners less sharp, what then? Who is going to invent the next generation of technology for them to steal from?
We need to reward the people who dare to create something new and succeed in making it happen. Letting others copy their work and dilute their success diminishes their reward, often to the point of making it pointless to even attempt something equivalent in the future.
The future you espouse is all wonderful so long as you’re the pointless conman who never invents anything and rips everyone off. When there’s nobody left to rip off the house of cards will come crashing down.
Anything that is related to the infrastructure of the country should be DISCONNECTED from the internet and a live human being should have the job of controlling the system. This would make it more secure and create some good jobs for people who could use the work.
Power, gas, pipelines, water, etc… All should be offline!
Remember stuxnet. Those computers were completely disconnected, and all that did was slow the hack. Any computer needs to be updated and maintained, and that means transferring information to that computer in some fashion. Hack the source of that information, plant your code, and the fact that its not on the ‘net is just an inconvenience.
Without a doubt this is the future of warfare. Everyone is worried about traditional bombs and physical attacks. Instead, we will wake up one day to find our electricity turned off, or our infrastructure under someone elses control. If we dont prepare now , we are in for a rude awakening.
use http://www.surfpatrol.ru/report check ur system for vulnerabilities with SurfPatrol and update it in time